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ABSTRACT 



The present invention includes video image control means 
for acquiring information for a plurality of video images 
from a video image source, cursor position control means for 
calculating cursor position information and generating cur- 
sor image information, display image generating means for 
synthesizing a plurality of video images and cursor image 
information and displaying thejsame on a display device, 
(g^ance,informaUon,genexating me^^for pher^ ting>disj 
(tance\.informatipri hased on video image poarion^nforma- 
\Jion and- curar. positioning 

^control means.for deciding volumeof audio for_ the,plurality 
^of_video. images based on this^distahce information,^and^ 
^making-output -to-an audio^ outpurdevice.. 
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MULTICHANNEL INFORMATION PROCESSING 
DEVICE 

[0001] This is a continuation of International Application 
PCT/JP01/02423, with an international filing date of Mar. 
26, 2001. 

TECHNICAL FIELD 

[0002] The present invention relates to a multichannel 
information processing device capable of simultaneously 
displaying on a display device a plurality of video images 
accompanied by audio data; more specifically, it relates to a 
multichannel information processing device that enables a 
user to easily apprehend content of a video image specified 
from among a plurality of video images displayed on a 
display device, a multichannel information processing 
method, a program for a multichannel information process- 
ing method, and a computer-readable recording medium on 
which is stored such program. 

BACKGROUND ART 

[0003] As high-speed Internet access using such means as 
ADSL and cable modems becomes increasingly common, 
users at home have greater opportunity to download over the 
Internet video images, such as television programs. In 
addition, interfaces between digital cameras and personal 
computers (PCs) have become more common, and users 
now commonly download video images they themselves 
shot into their PCs for processing. 

[0004] Further, as PC functions improve and hard disk 
capacity increases, large amounts of data can easily be 
stored on a hard disk, replayed and displayed; and an 
environment in which a plurality of video images are 
displayed on a display device can be easily accommodated. 
To present such two or more video information, it is possible 
to arrange two or more windows in arbitrary positions, and 
to display these video information simultaneously on a 
display, such as a CRT or a liquid crystal display. 

[0005] When a plurality of video images are thus dis- 
played, the audio data accompanying such video images is 
handled either by outputting none of the audio data or by 
synthesizing and outputting all the audio data at the same 
volume. When audio is not output, this audio data cannot be 
used for apprehending the content of a video image; as there 
is no audio to assist in apprehending the content of the video 
image, a user is forced to make a determination about video 
image content relying solely on the images displayed on the 
display device. When all the audio data is synthesized at the 
same volume, the various audio data interfere with each 
other, and are difficult to hear; it is also difficult to tell which 
audio belongs to which video image. 

[0006] It is an object of the present invention to facilitate 
recognition of which audio data corresponds to which video 
image, and to enable a user to easily apprehend the content 
of video image being displayed. 

DISCLOSURE OF THE INVENTION 

[0007] The multichannel information processing device 
according to a first aspect of the present invention is a 
multichannel information processing device capable of 
simultaneously displaying a plurality of video images on a 
display device, such device including video images control 



means for acquiring information for a plurality of video 
images, and for deciding video image position information 
relating to display position on the display device for the 
plurality of video images and then outputting the plurality of 
video images based on such video image position informa- 
tion; cursor position control means for calculating cursor 
position information based on cursor instructions informa- 
tion input via an input device and generating cursor images 
based on cursor position information and outputting the 
same; display image generating means for synthesizing and 
displaying the plurality of video images output by video 
image display control means and cursor ima ge inf ormation 
oujpjn^y^rs^position cormoJ_means; dist ance j nf ofma - 
Cfion genwajingjmeans foj^c^oUatmg^distahce between 
^d^playjpositions of the ~v¥ridus^video images and cursor 
(^display position based on video image L positidn~infonja^oni 
<,f^thVpluralityof videoimages and cursor position infor-_i 
^nation calculated by^cursor position contrpLmeans, and 
^generatmg distajo^ 
^eans - f6T^elSn^yo^ 

(pjurafity,of--video_images based on distance ~ info rma don 
S e ^^ t ^&^§&5?5i^'™? 011 gcnerajjng mea ns, and for \ 
£outputting~a!ibj o^a^ 

[0008] The multichannel information processing device 
according to a second aspect of the present invention is a 
multichannel infor matio n processing device according to the 
first aspect^herein audio^o^putfflntfol means sets volume^ 

(foT^udipior^ the video, image having ttie'«smallest distance 
^ajue.among-lhe^distance information generated by distance 

(mfdrmation generating means _at a predetermined ^sound^ 
wlume Va, and sets volume for~audio for the other video ^ 
Q^ges.aLprede^ermined sound, volume Vb^which is lower 

< tfaan Va ; 

[0009] The multichannel information processing device 
according to a third aspect of the present invention is a 
multichannel information processing device according to the 
first aspect(whereirTaudio output control means sets volume^ 
^for audio data for a plurality of video images so as lo be in 1 
inverse proportion to distance _yalues_generated by distance 
in format ion-generating means. 

[0010] The multichannel information processing device 
according to a fourth aspect of the present invention is a 
multichannel information processing device according to the 
first aspect, wherein dista^^j^l^anon^enerat^ &y_dis^: 
^tanoFinformation generating means includes direction infor- 
matioajrelating to direction qf video, image display position 

(js^seen^from -cursor, display position, and audio output^ 
control means makes output to an audio output device based 
in distance information, so" that audio, for- the plurality of 

(video images is positioned in the sound image : space formed 

<Cby_the_audio outpuLdevice. 

[0011] The multichannel information processing device 
according to a fifth aspect of the present invention is a 
multichannel information processing device according to the 
first aspect, further including voice data recognition means 
for recognizing words included in audio for a plurality of 
video images and character information display means for 
converting words recognized by voice data recognition 
means into character data and displaying the same on a 
display device. 

[0012] The multichannel information processing device 
according to a sixth aspect of the present invention is a 
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multichannel information processing device according to the 
first aspect, including Internet connection means, web site 
search means for searching for related web sites on the 
Internet, using a word recognized by voice data recognition 
means as keyword, and web site display means for display- 
ing on a display device a web site found by web site search 
means. 

[0013] The multichannel information processing device 
according to a seventh aspect of the present invention is a 
multichannel information processing device according to the 
first aspect, further including video image selecting means 
for selecting, based on a prescribed algorithm, a specified 
video image from among a plurality of video images dis- 
played on a display device, wherein audio output control 
means outputs to an audio output device audio data for the 
video image selected by video image selecting means. 

[0014] The multichannel information processing device 
according to an eighth aspect of the present invention is a 
multichannel information processing device according to the 
seventh aspect, wherein video image selecting means 
switches to a different video image for selection whenever a 
prescribed length of time has passed. 

[0015] The multichannel information processing method 
according to a ninth aspect of the present invention is a 
multichannel information processing method for displaying 
a plurality of video images simultaneously on a display 
device, including a step for deciding display position on 
display device for video images to be displayed on the 
display device, a step for outputting information for the 
plurality of video images based on decided display position, 
a step for accepting cursor instructions information input 
from an input device, a step for calculating cursor position 
information for displaying cursor based on cursor position 
information, a step for generating cursor image information 
based on cursor instructions information, a step for synthe- 
sizing information for the plurality of video images and 
cursor image information, generating a display image, and 
displaying the same on the display device, a step for 
calculating distance between display position of the plurality 
of video images and display position of a cursor and 
generating distance information, and a step for deciding 
volume for audio for the plurality of video images based on 
distance information and outputting to an audio output 
device. 

[0016] The multichannel information processing method 
according to a tenth aspect of the present invention is a 
multichannel information processing method according to 
the ninth aspect, wherein volume for audio for the video 
image having the smallest distance value among the distance 
information generated for the plurality of video images is set 
at a predetermined sound volume Va, and volume for audio 
data for the other video images is set at predetermined sound 
volume Vb, which is lower than volume Va. 

[0017] The multichannel information processing method 
according to an eleventh aspect of the present invention is a 
multichannel information processing method according to 
the ninth aspect, wherein volume for audio for a plurality of 
video images is set so as to be in inverse proportion to 
distance values within distance information. 

[0018] The multichannel information processing method 
according to a twelfth aspect of the present invention is a 



multichannel information processing method according to 
the ninth aspect, further including a step for generating 
direction information relating to direction of display position 
of video images as seen from cursor display position, and a 
step for making output to an audio output device, so that 
audio data for the plurality of video images is positioned in 
the sound image space of the audio output device in accor- 
dance with distance information and direction information. 

[0019] The multichannel information processing method 
according to a thirteenth aspect of the present invention is a 
multichannel information processing method according to 
the ninth aspect, further including a step for recognizing 
words included in audio for a plurality of video images and 
a step for converting voice-recognized words into character 
data and displaying the same on a display device. 

[0020] The multichannel information processing method 
according to a fourteenth aspect of the present invention is 
a multichannel information processing method according to 
the thirteenth aspect, including a step for connecting to the 
Internet, a step for searching for related web sites on the 
Internet, using a voice-recognized word as keyword, and a 
step for displaying on a display device a found related web 
site. 

[0021] The multichannel information processing method 
according to a fifteenth aspect of the present invention is a 
multichannel information processing method according to 
the ninth aspect, further including a step for selecting with 
a prescribed algorithm a specified video image from among 
a plurality of video images displayed on a display device, 
and a step for outputting to an audio output device audio data 
for a selected video image. 

[0022] The program according to a sixteenth aspect of the 
present invention is a program for executing on a computer 
a multichannel information processing method for display- 
ing a plurality of video images simultaneously on a display 
device, such method including a step for deciding display 
position for video images on a display device, a step for 
outputting information for the plurality of video images 
based on decided display position, a step for accepting 
cursor instructions information input from an input device, 
a step for calculating cursor position information for dis- 
playing cursor based on cursor instructions information, a 
step for generating cursor image information based on 
cursor position information, a step for synthesizing infor- 
mation for the plurality of video images and cursor image 
information, generating a display image, and displaying the 
same on the display device, a step for calculating distance 
between display position of the plurality of video images 
and display position of a cursor and generating distance 
information, and a step for deciding volume for audio for the 
plurality of video images based on distance information and 
outputting to an audio output device. 

[0023] The program according to a seventeenth aspect of 
the present invention is a program for executing on a 
computer a multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, such method including a step for deciding 
display position for video images on a display device, a step 
for outputting information for the plurality of video images 
based on decided display position, a step for accepting 
cursor instructions information input from an input device, 
a step for calculating cursor position information for dis- 
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playing cursor based on cursor instructions information, a 
step for generating cursor image information based on 
cursor position informauon, a step for synthesizing infor- 
mation for the plurality of video images and cursor image 
information, generating a display image, and displaying the 
same on the display device, a step for calculating distance 
between display position of a plurality of video images and 
display position of a cursor and generating distance infor- 
mation, and a step for setting volume for audio for the video 
image having the smallest distance value among the distance 
informauon generated with regards to the plurality of video 
images at a predetermined sound volume Va, and setting 
volume for audio for the other video images at predeter- 
mined sound volume Vb, which is lower than Va, and 
outputting to an audio output device. 

[0024] The program according to an eighteenth aspect of 
the present invention is a program for executing on a 
computer a multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, such method including a step for deciding 
display device display position for video images on a display 
device, a step for outputting information for the plurality of 
video images based on decided display position, a step for 
accepting cursor instructions information input from an 
input device, a step for calculating cursor position informa- 
tion for displaying cursor based on cursor instructions 
information, a step for generating cursor image information 
based on cursor position information, a step for synthesizing 
information for the plurality of video images and cursor 
image information, generating a display image, and display- 
ing the same on the display device, a step for calculating 
distance between display position of the plurality of video 
images and display position of a cursor and generating 
distance information, and a step for setting volume for audio 
for the plurality video images so as to be in inverse propor- 
tion to distance value in distance information, and outputting 
to an audio output device. 

[0025] The program according to a nineteenth aspect of 
the present invention is a program for executing on a 
computer a multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, such method including a step for deciding 
display position for video images on a display device, a step 
for outputting information for the plurality of video images 
based on decided display position, a step for accepting 
cursor instructions information input from an input device, 
a step for calculating cursor position information for dis- 
playing cursor based on cursor instructions information, a 
step for generating cursor image information based on 
cursor position information, a step for synthesizing infor- 
mation for the plurality of video images and cursor image 
information, generating a display image, and displaying the 
same on the display device, a step for calculating distance 
between display position of the plurality of video images 
and display position of a cursor and generating distance 
information, a step for generating direction information 
relating to direction of display position for each video image 
as seen from cursor display position, and a step for output- 
ting to an audio output device so that audio data for the 
plurality of video images have set acoustic image positions 
in accordance with distance information and direction infor- 
mation in the sound image space in an audio output device. 



[0026] The program according to a twentieth aspect of the 
present invention is a program for executing on a computer 
a multichannel information processing method for display- 
ing a plurality of video images simultaneously on a display 
device, such method including a step for deciding display 
position for video images on a display device, a step for 
outputting information for the plurality of video images 
based on decided display position, a step for accepting 
cursor instructions information input from an input device, 
a step for calculating cursor position information for dis- 
playing cursor based on cursor instructions information, a 
step for generating cursor image information based on 
cursor position information, a step for voice-recognizing 
words included in audio data for the plurality of video 
images, and a step for synthesizing the plurality of video 
images, cursor image information and character informa- 
tion, generating a display image and displaying the same on 
a display device. 

[0027] The program according to a twenty-first aspect of 
the present invention is a program for executing on a 
computer a multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, such method including a step for deciding 
display position for video images to be displayed on a 
display device, a step for outputting information for the 
plurality of video images based on decided display position, 
a step for accepting cursor instructions information input 
from an input device, a step for calculating cursor position 
information for displaying cursor based on cursor position 
information, a step for generating cursor image information 
based on cursor position information, a step for calculating 
distance between position information for the plurality of 
video images and cursor position information and generating 
distance information, a step for selecting a specified video 
image from among the plurality of video images based on 
distance information and outputting audio data for this video 
image to an audio output device, a step for voice-recogniz- 
ing words included in audio data for output to an audio 
output device, a step for converting recognized words into 
character information and outputting the same, and a step for 
synthesizing the plurality of video images, cursor image 
information and character information, generating a display 
image and outputting the same to a display device. 

[0028] The program according to a twenty-second aspect 
of the present invention is a program for executing on a 
computer a multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, such method including a step for deciding 
display position for video images to be displayed on a 
display device, a step for outputting information for the 
plurality of video images based on decided display position, 
a step for accepting cursor instructions information input 
from an input device, a step for calculating cursor position 
information for displaying cursor based on cursor position 
information, a step for generating cursor image information 
based on cursor position information, a step for calculating 
distance between position information for the plurality of 
video images and cursor position information and generating 
distance information, a step for selecting a specified video 
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image from among the plurality of video images based on 
distance information and outputting audio data for this video 
image to the audio output device, a step for voice -recogniz- 
ing words included in audio data for output to the audio 
output device, a step for voice-recognizing words included 
in audio for the plurality of video images, a step for 
connecting to the Internet, a step for searching for related 
web sites on the Internet using a voice-recognized word as 
keyword, and a step for synthesizing the plurality of video 
images, cursor image information and a found web site, 
generating a display image and displaying the same on a 
display device. 

[0029] The program according to a twenty-third aspect of 
the present invention is a program for executing on a 
computer a multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, such method including a step for deciding 
display position for video images to be displayed on a 
display device, a step for outputting information for the 
plurality of video images based on decided display position, 
a step for selecting with a prescribed algorithm a specified 
video image from among the plurality of video images 
displayed on a display device, and a step for outputting 
audio for the selected video image to an audio output device. 

[0030] The recording medium according to a twenty- 
fourth aspect of the present invention is a computer-readable 
medium on which is recorded the programs of the sixteenth 
through twenty-third aspects. 

A BRIEF EXPLANATION OF THE DRAWINGS 

[0031] FIG. 1 is a block diagram showing a simplified 
configuration of the present invention. 

[0032] FIG. 2 shows an example of a video image position 
information table. 

[0033] FIG. 3 shows an example of screen configuration. 

[0034] FIG. 4 is a control block diagram for a first 
embodiment. 

[0035] FIG. 5 is a control flowchart for the first embodi- 
ment. 

[0036] FIG. 6 shows another example of a video image 
position information table. 

[0037] FIG. 7 is a control flowchart for a variation on the 
first embodiment 

[0038] FIG. 8 is a control block diagram for a second 
embodiment. 

[0039] FIG. 9 is a control flowchart for a variation on the 
second embodiment 

[0040] FIG. 10 is a control block diagram for a third 
embodiment. 

[0041] FIG. 11 is a control flowchart for the third embodi- 
ment. 

[0042] FIG. 12 is a control flowchart for a variation on the 
third embodiment 

[0043] FIG. 13 is a control block diagram for a fourth 
embodiment. 



[0044] FIG. 14 is a control flowchart for the fourth 
embodiment. 

PREFERRED EMBODIMENTS 
[0045] Basic Configuration 

[0046] FIG. 1 shows hardware configuration for an infor- 
mation processing device according to a first embodiment of 
the present invention. 

[0047] An information processing device 10, which may^ 
be:a:stanc^Iper|^ a PC body 1, ! a" 

display 2, such as a cathode ray tube or liquid crystal display* 
device, speaker 3, input device 4, such as a mouse, trackball, 
touchpad, or keyboard, and a modem 5 capable of connect-, 
ing to the Internet. The modem 5 may be, for example, an? 
ADSL modem, which connects to the Internet over public 
lines. 

[0048] The PC body 1 is configured so as to allow con- ^ 
nection to an ISP (Internet Service Provider) 20 using thef 
modem 5 and a public phone line and download a plurality 
of video images informajiojHxom^strearjiing-data available 
at the ISP 20. TEeJdoxvnloaa^ arcj, 
( ^p^ ycg^n~^p6sitiojs^n Tttje 'djspjay_2^tb^haye^ beea 
progr^n^ed.m.advance^llie PC body 1 may of course be j 
configured so that a user sets display position using tools 3 
containing in a program. [] 

[0049] FIG. 2 shows a position information table relating 
to video image display position. 

[0050] This position information table has entries for 
channel number, center position, width, height, and the like; 
it contains information regarding center coordinates (CXi, 
CYi), width Wi and height Hi for video images from each 
channel number CHi (where i=l to n). The position infor- 
mation table is set up for every (n) channels to be displayed, 
and is prepared for the maximum number of channels that 
the PC body 1 can process or the display 2 can display. 

[0051] FIG. 3 shows an example of a screen displayed in 
a case where video images are displayed based on position 
information set according to such a position information 
table. In the example shown, there are three channels to be 
displayed; the display position of each video image is 
decided by their respective coordinates (CXi, CYi), width 
Wi and height Hi. 

[0052] First Embodiment 

[0053] FIG. 4 shows a control block diagram of a first 
embodiment of the present invention. 

[0054] TEe^fonlE^ 

^o^iogmSg£^^ -for acquiring^ 

c^gjug^y^f^^ 
Video im£ge~llisplay control means 101, for example, 
decides display position for each video image based on a 
position information table as shown in FIG. 2, and outputs 
this video image display position information and the video 
images. 

[0055] A cursor position deciding means 102 accepts 
cursor instructions information from an input device 140, 
which may be a mouse, trackball, touchpad, keyboard or 
other pointing device, and it calculates information for next 



04/19/2004, EAST Version: 1.4.1 



US 2004/0056885 Al Mar. 25, 2004 



cursor position based on this cursor instructions information 
and current cursor position information, and outputs the 
same. 

[0056] Cursor position information output from cursor 
position deciding means 102 is input into cursor image 
generating means 103. Cursor image generating means 103 
generates cursor image information based on cursor position 
information, and outputs the same. 

[0057] Video image information and video image position 
information output from video image display control means 
101 and cursor image information output from generating 
means 103 are input into display image generating means 
104. Display image generating means 104, which includes 
an image output buffer such as VRAM, arranges video 
images based on their respective position information, and 
synthesizes a cursor image, outputting this to a display 
device 120. 

[0058] Video image position information output from 
video image display control means 101 and cursor position 
information output from cursor position deciding means 102 
are input mto ^disl ancerirjfor^ 

Distance information generating means lOS^alculates dis- 
tance between display position of each video image and 
cursor display position, based on video image position 
information and cursor position information. The distance 
between the center position of a video image and the center 
position of the cursor may be used as distance between a 
video image display position and cursor display position. In 
addition, distance can be set to be zero when a cursor is 
positioned within a video image window. 

[0059] Distance information generated by distance infor- 
mation generating means 105 is input into video image 
selecting means 10^JViBeo_im^e^selectmg-means'^l063 ? 
rdecide^lhX^ audio will 'bejoutput / 

^ based on distance information g eneraleU jy distanc j^infor^y 
{matj on generating means^lOS^For example, the video image 
having the smallest distance value among the distance 
information for video images can be selected. 

[0060] ^Video_image selecting j^eai£ri06~outputs-audio 
^utput^lecuon,ipfpimatioii,bas ed^ n the selected-video- 
^imTgeJg_audjo^oufo^ Audio ojitpUt^ 

r ^electin g^Teans 1071selectively acquires aumo'datFfor the 
^video~image7~as set according to audio output selection^ 
information, from among the plurality of video images fromO 
(the video image source-200,-and_ outputs^ thisjo an audio^ 
outp ut dev ice,130.- 

[0061] Operations using this configuration will be 
explained based on the flowchart of FIG. 5. 

[0062] In step Sll, variable (i) is initialized. In step S12, 
cursor position information is read. In step S13, it is deter- 
mined whether cursor display information is located within 
a video image display frame for channel number CHi. A 
video image position information table may include, in 
addition to entries for channel number, center position, 
display width and display height as shown in FIG. 6, entries 
for neighborhood width and neighborhood height. When a 
cursor display position is located within the range stipulated 
by center position coordinates (CXi, CYi) of video image of 
channel number CHi and neighborhood width NWi and 
neighborhood height NHi, it is determined to be within the 
display frame for the respective video image. In such a case, 



the display frame may be set within a range of a prescribed 
distance around a window displaying a video image, and 
when a cursor is located within such frame, determination is 
made that the respective video image has been selected. 
When it has been determined that a cursor display position 
is located within a display frame for a video image for 
channel number CHi, control proceeds to step S16. 

[0063] In step SI 6, audio data for the video image of 
channel number CHi is output to the audio output device 
130. 

[0064] If, in step S13, it is determined that a cursor display 
position is not located within a display frame for a video 
image for channel number CHi, control proceeds to step 
S14. 

[0065] In step S14, it is determined whether variable (i) 
has reached number of video images displayed (n). In case 
where variable (i) equals number of display channels n, 
control returns to step Sll; in all other cases, control 
proceeds to step S15. 

[0066] In step S15, variable (i) is incremented by 1 and 
control returns to step S13. 

[0067] With such a configuration, even if the display 
position of a mouse cursor is not on a video image window, 
but is in the neighborhood of a window, the audio for the 
respective video image is output. Thus a user can use a 
pointing device such as mouse or trackball to move a cursor 
so that it approaches the window displaying the desired 
video image, allowing such user to hear the audio and thus 
easily apprehend the content of the video image. 

[0068] Variation 1 

[0069] As an alternative configuration, the video image 
closest to cursor display position is selected for audio output. 
FIG. 7 shows a flowchart in such a case. 

[0070] In step S21, variable (i) is initialized. In step S22, 
cursor position information is read. In step S23, distance (di) 
between cursor display position and display position for the 
video image of channel number CHi is calculated. This 
distance (di) may be the distance between the central coor- 
dinates of a window displaying a video image and the central 
coordinates of a cursor; alternatively, it may be the shortest 
distance between a frame of a window displaying a video 
image and the central coordinates of a cursor. 

[0071] In step S24, it is determined whether variable (i) 
has reached number of video images being displayed (n). In 
case where variable (i) equals number of display channel (n), 
control proceeds to step S26; in all other cases, control 
proceeds to step S25. 

[0072] In step S25, variable (i) is incremented by 1 and 
control proceeds to step S26. 

[0073] In step S26, the smallest value (di) between a video 
image display position and a cursor display position is 
determined, and the video image for the relevant channel 
CHi is selected as a selected image. 

[0074] In step S27, audio for the selected video image is 
output. 

[0075] With such a configuration, there is no need to 
prepare a table for providing a display frame for the neigh - 
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borhood of a window displaying a video image, as audio for 
the video image closest to the cursor display position is 
output. 

[0076] Variation 2 

[0077] An alternative configuration is, for example, to 
change the color of display window frame for a video image 
for which audio is being output, and to change the tone of 
video images not selected, allowing identification of which 
channel has been selected. 

[0078] Variation 3 

[0079] In another alternative configuration, audio data for 
each video image is amplified in inverse proportion to 
distance (di) between video image display position and 
cursor display position, synthesized and output. 

[0080] Calculation of distance (di) between cursor display 
position central coordinates (CuX, CuY) and video image 
display position central coordinates (CXi, CYi) can be 
calculated according to the following formula: 

dim&CuX-CXif+iCuY-CYif) 1/2 . 

[0081] Using this distance (di), the amplification rate 
CHig[dB] for audio data for the corresponding video image 
is set at: CHig[dB]=a*(di) (where a is an constant greater 
thanO). 

[0082] With such a configuration, audio for a video image 
near cursor display position is output at high volume, and 
audio for a video image far from cursor display position is 
output at low volume. Accordingly, a user can use a mouse 
or other pointing device to increase the volume of audio for 
a video image approached by a cursor, allowing such user to 
apprehend the content of such video image; meanwhile, 
while audio volume for other video images is low, that audio 
is still being output, assisting a user in apprehending the 
content of the respective video images. 

[0083] Variation 4 

[0084] In yet another alternative configuration, direction 
information relating to direction of video image display 
position as seen from the cursor display position is calcu- 
lated, and based on direction information and distance 
information, audio data for a video image is positioned in the 
sound space generated by the audio output device 130. 

[0085] As described above, both cursor display position 
and video image display position can be expressed as 
display device coordinates; these coordinates can be used to 
calculate direction information for display position of a 
video image as seen from the cursor. Using such direction 
information and distance information (di), acoustic images 
for audio data can be assigned positions within the sound 
space generated by the audio output device 130. In such 
cases, however, the audio output device 130 must have at 
least 2 speakers, and the output from each speaker must be 
controlled so that a sound space is generated by the audio 
output from the speakers. 

[0086] For example, for cursor central coordinates (CuX, 
CuY) and channel CHi central coordinates (CXi, CYi), 
x-axis direction CXid is calculated according to the follow- 
ing formula: 

CXid-CuX-CXL 



[0087] Using this x-axis direction CXid, volume for left 
and right speakers is decided and audio output accordingly, 
and it is possible to orientate acoustic image. 

[0088] With such a configuration, the display position of a 
video image on the display device 120 and the position of 
audio output from the audio output device 130 can be made 
to match, facilitating apprehension of video image content. 

[0089] Second Embodiment 

[0090] FIG. 8 is a control block diagram of a second 
embodiment of the present invention. 

[0091] The information processing device 150 includes 
video image display control means 101 for acquiring infor- 
mation for a plurality of video images from a video image 
source 200. As in the first embodiment, this video image 
display control means 101 decides display position for each 
video image in accordance with a preset position informa- 
tion table, and outputs this video image position information 
and video image information. 

[0092] As in the first embodiment, display image gener- 
ating means 104, which includes an image output buffer 
such as VRAM, arranges a plurality of video images based 
on their respective position information and outputs the 
images. 

[0093] Video image position information from the video 
image display control means 101 is input into video image 
selecting means 108. Video image selecting means 108 has 
an algorithm for selecting video image information from a 
plurality of channels at predetermined periods. For example, 
one channel number is selected and audio selection infor- 
mation for selection of audio for the corresponding video 
image is output; whenever a certain amount of time has 
elapsed, the channel number selected switches according to 
channel number order. 

[0094] Audio output selection information output from 
video image selecting means 108 is input into audio outgut 
selecting means 107. Based on audio output selection infor- 
mation, audio output selecting means 107 outputs to [the 
audio output device 130 audio data for a video image of a " 
selected channel. \^ 

[0095] FIG. 9 shows a flowchart for an algorithm config- 
ured so that video images displayed are selected in order, 
and the audio data for a selected video image is output for 
a set length of time. 

[0096] In step S31, variable (i) is initialized. In step S32, 
a timer is initialized and begins counting elapsed time. This 
timer times output of audio for the video image of a selected 
channel. 

[0097] In step S33, audio data for the video image of 
channel CHi is output to the audio output device 130. 

[0098] In step S34, it is determined whether timer count 
value has reached a predetermined value. If it is determined 
that the timer count value has reached a predetermined 
value, control proceeds to step S35. 

[0099] In step S35, variable (i) is incremented by 1, and 
the video image for the next channel number is selected. 

[0100] In step S36, it is determined whether the value of 
variable (i) has exceeded number of channels being dis- 
played (n). If it is determined that the value of variable (i) 
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has not exceeded number of channels (n), control returns to 
step S32; if it is determined that the value of variable (i) has 
exceeded number of channels (n), control returns to step 
S31. 

[0101] With a configuration in which video image select- 
ing means 108 selects video image in accordance with such 
an algorithm and audio for such selected video image is 
output, video image selected is periodically switched, and 
the audio for the respective video image is output for a set 
period of time. Without having to designate video image 
with a pointing device or the like, a user can periodically 
hear the audio for each video image, facilitating apprehen- 
sion of the content of the video images. 

[0102] Third Embodiment 

[0103] FIG. 10 is a control block diagram for a third 
embodiment of the present invention. 

[0104] This information processing device 160 includes 
video image display control means 101 for acquiring infor- 
mation for a plurality of video images from the video image 
source 200. As in the first embodiment, video image display 
control means 101 decides display position for each video 
image and outputs this video image position information and 
the video image itself. 

[0105] Cursor position deciding means 102 receives cur- 
sor instructions information input from the input device 140, 
which may be a mouse, trackball, touchpad, keyboard or 
other pointing device, calculates next cursor position infor- 
mation based on these cursor instructions information and 
current cursor display position, and outputs the same. 

[0106] Cursor position information output from cursor 
position deciding means 102 is input into cursor image 
generating means 103. Cursor image generating means 103 
generates cursor image information based on cursor position 
information, and outputs the same. 

[0107] Video image position information output from 
video image display control means 101 and cursor position 
information output from cursor position deciding means 102 
are input into distance information generating means 105, 
Distance information generating means 105 calculates dis- 
tance between each video image display position and cursor 
display position based on video image position information 
and cursor position information. The distance between the 
center position of a video image and the center position of 
the cursor may be used as distance between a video image 
display position and cursor display position. In addition, 
distance can be set to be zero when a cursor is positioned 
within a video image window. 

[0108] Distance information generated by distance infor- 
mation generating means 105 is input into video image 
selecting means 106. Video image selecting means 106 
decides the video image for which audio will be output 
based on distance information generated by distance infor- 
mation generating means 105. For example, the video image 
having the smallest distance value among the distance 
information for video images can be selected. 

[0109] Video image selecting means 106 outputs audio 
output selection information based on the selected video 
image to audio output selecting means 107. Audio output 
selecting means 107 selectively acquires audio data for the 
video image, as set according to audio output selection 



information, from among the plurality of video images from 
the video image source 200, and outputs this to the audio 
output device 130. 

[0110] Audio data output from audio output selecting 
means 107 is input into voice recognition means 109. Voice 
recognition means 109 recognizes and extracts words having 
meaning from the audio data, and converts these to text 
information. 

[0111] Text information converted by voice recognition 
means 109 is input into recognized character display means 
110. Recognized character display means 110 generates 
character data for display on the display device 120 based on 
character information output from voice recognition means 
109 and outputs the same to display image generating means 
104. 

[0112] Video image information and video image position 
information output from video image display control means 
101, cursor image information output from cursor image 
generating means 103, and character data output from rec- 
ognized character display means 110 are input into display 
image generation means 104. Display image generation 
means 104, which includes an image output buffer such as 
VRAM, arranges the plurality of video images based on 
their respective position information, synthesizes cursor 
image and character data, and outputs the same to the 
display device 120. 

[0113] The character data output from recognized charac- 
ter display means 110 can be configured so as to be dis- 
played in the neighborhood of the corresponding video 
image, and may also be configured so that information 
stating which video image the character data comes from is 
displayed on the display device 120. 

[0114] In an alternative configuration, character data 
acquired from audio data for the video image of a selected 
channel is displayed, but audio is not output from the audio 
output device 130. 

[0115] Operations in such a case will be explained with 
reference made to the flowchart of FIG. 11. 

[0116] In step S41, variable (i) is initialized. In step S42, 
cursor position information is read. In step S43, it is deter- 
mined whether or not cursor display position is located 
within the display frame for video image of channel number 
CHi. In this case, as in the first embodiment, display frame 
may be set as the range defined by a predetermined distance 
around a window in which a video image is being displayed; 
when a cursor is located within such a display frame, 
determination is made that the relevant video image has 
been selected. In the case where it is determined that a cursor 
display position is located within the display frame for video 
image of channel Chi, control proceeds to step S44. 

[0117] In step S44, audio data for the video image of 
channel number CHi is output to the audio output device 
130. 

[0118] In step S45, words having meaning in audio data 
for the video image of channel number CHi are recognized 
and converted into character data. 

[0119] In step S46, recognized character data is sent to 
display image generation means 104 and displayed on the 
display device 120. 
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[0120] If in step S43 it is determined that cursor display 
position is not located within a display frame of the video 
image for channel number CHi, control proceeds to step 
S47. 

[0121] In step S47, it is determined whether the value of 
variable (i) has reached number of channels being displayed 
(n). If it is determined that the value of variable (i) is equal 
to number of channels n, control returns to step S41; in all 
other cases, control proceeds to step S48. 

[0122] In step S48, the variable (i) is incremented by 1, 
and then control returns to step S43. 

[0123] Such a configuration enables visual recognition of 
words included in audio data for the video image of a 
selected channel, facilitating apprehension of the contents of 
the video image. 

[0124] Variation 

[0125] FIG. 12 is a block diagram of a variation on the 
third embodiment. 

[0126] This information processing device 170 is provided 
with voice recognition means 109 and recognized character 
display means 110 in the same number as the number of 
channels to be displayed; words with meaning are extracted 
from the audio data for the video image in each respective 
channel, converted to character data, input into display 
image generation means 104 and displayed on the display 
device 120. 

[0127] The character data may be displayed in the neigh- 
borhood of the display position of the corresponding video 
image, or the character data alone may be displayed on a list. 

[0128] With such a configuration, words extracted from 
audio data for each video image are displayed on the display 
device 120, facilitating a user's apprehension of the content 
of each video image. 

[0129] Fourth Embodiment 

[0130] FIG. 13 is a control block diagram of a fourth 
embodiment of the present invention. 

[0131] This information processing device 180 includes 
video image display control means 101 for acquiring infor- 
mation for a plurality of video images from the video image 
source 200. As in the first embodiment, video image display 
control means 101 decides display position for each video 
image, and outputs this video image position information 
and video image information. 

[0132] Cursor position deciding means 102 accepts cursor 
instructions input from an input device 140, which may be 
a mouse, trackball, touchpad, keyboard or other pointing 
device, and it calculates information for next cursor position 
based on this cursor position information and current cursor 
instructions input, and outputs the same. 

[0133] Cursor position information output from cursor 
position deciding means 102 is input into cursor image 
generating means 103. Cursor image generating means 103 
generates cursor image information based on cursor position 
information, and outputs the same. 

[0134] Video image position information output from 
video image display control means 101 and cursor position 
information output from cursor position deciding means 102 



are input into distance information generating means 105. 
Distance information generating means 105 calculates dis- 
tance between display position of each video image and 
cursor display position, based on video image position 
information and cursor position information. The distance 
between the center position of a video image and the center 
position of the cursor may be used as distance between a 
video image display position and cursor display position. In 
addition, distance can be set to be zero when a cursor is 
positioned within a video image window. 

[0135] Distance information generated by distance infor- 
mation generating means 105 is input into video image 
selecting means 106. Video image selecting means 106 
decides the video image for which audio will be output 
based on distance information generated by distance infor- 
mation generating means 105. For example, the video image 
having the smallest distance value among the distance 
information for video images can be selected. 

[0136] The video image selecting means 106 outputs 
audio output selection information based on the selected 
video image to audio output selecting means 107. Audio 
output selecting means 107 selectively acquires audio data 
for the video image, as set according to audio output 
selection information, from among the plurality of video 
images from the video image source 200, and outputs this to 
the audio output device 130. 

[0137] Audio data output from audio output selecting 
means 107 is input into voice recognition means 109. Voice 
recognition means 109 recognizes, extracts, and converts 
into character information words with meaning in input 
audio. 

[0138] Text information converted by voice recognition 
means 109 is input into recognized character display means 
110. Recognized character display means 110 generates 
character data to be displayed on the display device 120 
based on character information input from voice recognition 
means 109 and outputs the same to display image generating 
means 104. 

[0139] Text information output from voice recognition 
means 109 is input into web search means 111. Web search 
means 111 accesses the Internet via Internet connection 
means 113, and searches for searchable web sites on the 
Internet using character information acquired from voice 
recognition means 109 as keywords. For web site searching, 
connection may be made to a search engine site such as 
Yahoo, Goo or Google, and search results received there- 
from. Search results are received by web search means 111 
via Internet connection means 113 and sent to web site 
display means 112. 

[0140] Web site display means 112 accesses the URL of a 
web site obtained from the search results, acquires informa- 
tion from the web site and outputs the same to display image 
generating means 104. Web site display means 112 may be 
a browser such as Internet Explorer or Netscape Navigator. 
Web site information may also be acquired by accessing the 
Internet using Internet connection means 113. 

[0141] Video image information and video image position 
information output from video image display control means 
101, cursor image information output from cursor image 
generating means 103, character data output from recog- 
nized character display means 110, and web site information 
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output from web site display means 112 are input into 
display image generating means 104. Display image gener- 
ating means 104, which includes an image output buffer 
such as VRAM, arranges the plurality of video images based 
on their respective position information, synthesizes cursor 
image, character data and web site information, and outputs 
the same to the display device 120. 

[0142] The operations for such a configuration will be 
explained with reference made to the flowchart of FIG. 14. 
In step S51, variable (i) is initialized. In step S52, cursor 
position information is read. In step S53, it is determined 
whether or not cursor display position is located within the 
display frame for video image of channel number CHi. Id 
this case, as in the first embodiment, display frame may be 
set as the range defined by a predetermined distance around 
a window in which a video image is being displayed; when 
a cursor is located within such a display frame, determina- 
tion is made that the relevant video image has been selected. 
When determination is made that cursor display position is 
located within the display frame for the video image of 
channel number CHi, control proceeds to step S54. 

[0143] In step S54, audio data for the video image of 
channel number CHi is output to the audio output device 
130. 

[0144] In step S55, words having meaning in audio data 
for the video image of channel number CHi are recognized 
and converted into character data. 

[0145] In step S56, recognized character data is sent to 
display image generating means 104 and displayed on the 
display device 120. 

[0146] In step S57, a web search is carried out using 
character data obtained through voice recognition. 

[0147] In step S58, web site information is acquired based 
on results of the web search and displayed on the display 
device 120. 

[0148] If, in step S53, it is determined that cursor display 
position is not located within a display frame of the video 
image for channel number CHi, control proceeds to step 
S59. 

[0149] In step S59, it is determined whether the value of 
variable (i) has reached number of channels being displayed 
(n). If it is determined that the value of variable (i) is equal 
to number of channels n, control returns to step S51; in all 
other cases, control proceeds to step S60. 

[0150] In step S60, the variable (i) is incremented by 1, 
and then control returns to step S53. 

[0151] With such a configuration, words obtained from 
audio data for a video image are used to acquire information 
from related web sites, which is displayed on a display 
device. This facilitates acquisition of information relating to 
each video image and simplifies acquisition of a variety of 
data. 

INDUSTRIAL APPLICABILITY 

[0152] In accordance with the present invention, the audio 
data corresponding to a specific video image from among a 
plurality of video images being displayed on a display 
device can be easily identified, facilitating a user's appre- 
hension of the contents of that video image. 



[0153] While only selected embodiments have been cho- 
sen to illustrate the present invention, to those skilled in the 
art it will be apparent from this disclosure that various 
changes and modifications can be made herein without 
departing from the scope of the invention as defined in the 
appended claims. Furthermore, the foregoing description of 
the embodiments according to the present invention is 
provided for illustration only, and not for the purpose of 
limiting the invention as defined by the appended claims and 
their equivalents. 

1. A multichannel information processing device wherein 
a plurality of video images are displayed simultaneously on 
a display device, comprising: 

video image information control means for acquiring 
information for said plurality of video images, and for 
deciding video image position information relating to 
display position on a display device for said plurality of 
video images and outputting said information for a 
plurality of video images based on said video image 
position information; 

cursor position control means for calculating cursor posi- 
tion information based on cursor instructions informa- 
tion input via an input device and generating and 
outputting cursor image information based on said 
cursor position information; 

display image generating means for synthesizing infor- 
mation for the plurality of video images output by said 
video image display control means and cursor image 
information output by cursor position control means 
and displaying the same on said display device; 

distance information generating means for calculating 
distance between display positions of said plurality of 
video images and cursor display position based on 
video image position information for said plurality of 
video images and cursor position information calcu- 
lated by said cursor position information control 
means, and generating distance information; and 

audio output control means for deciding volume of audio 
data for said plurality of video images based on dis- 
tance information generated by said distance informa- 
tion generating means, and outputting audio data to an 
output device. 

2. A multichannel information processing device accord- 
ing to claim 1, wherein said audio output control means sets 
volume for audio data for the video image having the 
smallest distance value among the distance information 
generated by said distance information generating means at 
a predetermined sound volume Va, and sets volume for 
audio corresponding to the other video images at predeter- 
mined sound volume Vb, which is lower than said volume 
Va. 

3. A multichannel information processing device accord- 
ing to claim 1, wherein said audio output control means sets 
volume for audio data for said plurality of video images so 
as to be in inverse proportion to distance values generated by 
said distance information generating means. 

4. A multichannel information processing device accord- 
ing to claim 1, wherein distance information generated by 
said distance information generating means includes direc- 
tion information relating to direction of video image display 
position as seen from cursor display position, and said audio 
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output control means makes output to an audio output device 
based on said distance information, so that audio data for 
said plurality of video images is positioned in the sound 
space formed by said audio output device. 

5. A multichannel information processing device accord- 
ing to claim 1, further comprising: 

voice data recognition means for recognizing words 
included in audio data for said plurality of video 
images, and 

character information display means for converting words 
recognized by said voice data recognition means into 
character data and displaying the same on said display 
device. 

6. A multichannel information processing device accord- 
ing to claim 5, comprising: 

Internet connection means, 

web site search means for searching for related web sites 
on the Internet, using a word recognized by said voice 
data recognition means as keyword; and 

web site display means for displaying on said display 
device a web site found by said web site search means. 

7. A multichannel information processing device accord- 
ing to claim 1, further including video image selecting 
means for selecting, based on a prescribed algorithm, a 
specified video image from among a plurality of video 
images displayed on said display device, wherein said audio 
output control means outputs to an audio output device 
audio data for the video image selected by said video image 
selecting means. 

8. A multichannel information processing device accord- 
ing to claim 7, wherein said video image selecting means 
switches to a different video image for selection whenever a 
prescribed length of time has passed. 

9. A multichannel information processing method for 
displaying a plurality of video images simultaneously on a 
display device, including: 

a step for deciding display position on said display device 
for video images to be displayed on a display device; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 

a step for synthesizing information for said plurality of 
video images and said cursor image information, gen- 
erating a display image, and displaying the same on 
said display device; 

a step for calculating distance between display position of 
said plurality of video images and said cursor display 
position and generating distance information; and 

a step for deciding volume for audio for the plurality of 
video images based on said distance information and 
outputting to an audio output device accordingly. 



10. A multichannel information processing method 
according to claim 9, wherein volume for audio data for the 
video image having the smallest distance value among the 
distance information for a plurality of video images is set at 
a predetermined sound volume Va, and volume for audio 
data for the other video images is set at predetermined sound 
volume Vb, which is lower than volume Va. 

11. A multichannel information processing method 
according to claim 9, wherein volume for audio data for said 
plurality of video images is set so as to be in inverse 
proportion to distance values within distance information. 

12. A multichannel information processing method 
according to claim 9, further comprising: 

a step for generating direction information relating to 
direction of display position of video images as seen 
from cursor display position, and 

a step for making output to said audio output device, so 
that audio data for said plurality of video images is 
positioned in the sound space of the audio output 
device in accordance with said distance information 
and said direction information. 

13. A multichannel information processing method 
according to claim 9, further comprising: 

a step for voice-recognizing words included in audio for 
said plurality of video images, and 

a step for converting voice-recognized words into char- 
acter data and displaying the same on said display 
device. 

14. A multichannel information processing method 
according to claim 13, comprising: 

a step for connecting to the Internet; 

a step for searching for related web sites on the Internet, 
using a voice-recognized word as keyword; and 

a step for displaying on said display device a found related 
web site. 

15. A multichannel information processing method 
according to claim 9, further comprising: 

a step for selecting with a prescribed algorithm a specified 
video image from among said plurality of video images 
displayed on said display device, and 

a step for outputting to said audio output device audio data 
for a selected video image. 

16. A program for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 
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a step for synthesizing information for said plurality of 
video images and said cursor image information, gen- 
erating a display image, and displaying the same on 
said display device; 

a step for calculating distance between display position of 
said plurality of video images and display position of 
said cursor and generating distance information; and 

a step for deciding volume for audio for said plurality of 
video images based on said distance information and 
outputting to an audio output device accordingly. 

17. A program for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 

a step for synthesizing information for said plurality of 
video images and said cursor image information, gen- 
erating a display image, and displaying the same on 
said display device; 

a step for calculating distance between display position of 
said plurality of video images and display position of 
said cursor and generating distance information; and 

a step for setting volume for audio data for the video 
image having the smallest distance value among the 
distance information generated with regard to said 
plurality of video images at a predetermined sound 
volume Va, and setting volume for audio for the other 
video images at predetermined sound volume Vb, 
which is lower than volume Va, and outputting to an 
audio output device accordingly. 

18. A program for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 



a step for synthesizing information for said plurality of 
video images and said cursor image information, gen- 
erating a display image, and displaying the same on 
said display device; 

a step for calculating distance between display position of 
said plurality of video images and display position of 
said cursor and generating distance information; and 

a step for setting volume for audio data for said plurality 
of video images so as to be in inverse proportion to 
distance value in distance information, and outputting 
to an audio output device accordingly. 

19. Aprogram for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 

a step for synthesizing information for said plurality of 
video images and said cursor image information, gen- 
erating a display image, and displaying the same on 
said display device; 

a step for calculating distance between display position of 
said plurality of video images and display position of 
said cursor and generating distance information; 

a step for generating direction information relating to 
direction of display position for each video image as 
seen from cursor display position; and 

a step for making output to said audio output device so 
that audio data corresponding to said plurality of video 
images is positioned at acoustic image positions in the 
sound space of said audio output device in accordance 
with said distance information and said direction infor- 
mation. 

20. Aprogram for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 
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a step for generating cursor image information based on 
said cursor position information; 

a step for voice-recognizing words included in audio data 
for said plurality of video images; 

a step for converting voice-recognized words into char- 
acter data and outputting the same; and 

a step for synthesizing said plurality of video images, said 
cursor image information and said character informa- 
tion, generating a display image and displaying the 
same on said display device. 

21. A program for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 

a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 

a step for calculating distance between position informa- 
tion for said plurality of video images and said cursor 
position information and generating distance informa- 
tion; 

a step for selecting a specified video image from among 
the plurality of video images based on said distance 
information and outputting audio data for this video 
image to an audio output device; 

a step for voice-recognizing words included in audio data 
for output to said audio output device; 

a step for converting voice- recognized words into char- 
acter information and outputting the same; and 

a step for synthesizing said plurality of video images, said 
cursor image information and said character informa- 
tion, generating a display image and outputting the 
same to a display device. 

22. Aprogram for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for accepting cursor instructions information input 
from said input device; 



a step for calculating cursor position information for 
displaying cursor based on said cursor instructions 
information; 

a step for generating cursor image information based on 
said cursor position information; 

a step for calculating distance between position informa- 
tion for said plurality of video images and said cursor 
position information and generating distance informa- 
tion; 

a step for selecting a specified video image from among 
the plurality of video images based on said distance 
information and outputting audio data for this video 
image to an audio output device; 

a step for voice-recognizing words included in audio data 
for the plurality of video images; 

a step for connecting to the Internet; 

a step for searching for related web sites on the Internet 
using a voice-recognized word as keyword; and 

a step for synthesizing said plurality of video images, said 
cursor image information and said found web site, 
generating a display image and displaying the same on 
said display device. 
23. Aprogram for executing on a computer a multichannel 
information processing method for displaying a plurality of 
video images simultaneously on a display device, such 
method comprising: 

a step for deciding display position on display device for 
said video images to be displayed; 

a step for outputting information for said plurality of 
video images based on decided display position; 

a step for selecting with a prescribed algorithm a specified 
video image from among said plurality of video images 
displayed on said display device; and 



a step for outputting audio data for the selected video 
image to said audio output device. 



24. A computer-readable 


medium 


on 


which 


is 


recorded 


the program of claim 16. 












25. A computer-readable 


medium 


on 


which 


is 


recorded 


the program of claim 17. 












26. A computer-readable 


medium 


on 


which 


is 


recorded 


the program of claim 18. 












27. A computer-readable 


medium 


on 
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is 


recorded 


the program of claim 19. 












28. A computer-readable 


medium 
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is 


recorded 


the program of claim 20. 












29. A computer- readable 


medium 


on 


which 


is 


recorded 


the program of claim 21. 












30. A computer- readable 


medium 


on 


which 


is 


recorded 


the program of claim 22. 












31. A computer-readable 


medium 


on 


which 


is 


recorded 



the program of claim 23. 

* * * * * 
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